Multilevel Stochastic Gradient Methods for Nested Composition Optimization

Duality-free Methods for Stochastic Composition Optimization

We consider the composition optimization with two expected-value functions in the form of 1 n ∑n i=1 Fi( 1 m ∑m j=1Gj(x))+ R(x), which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement learning and nonlinear embedding. Full Gradient or classical stochastic gradient descent based optimization algorithms are unsuitab...

Multilevel Conjugate Gradient Methods

Stochastic gradient methods for the optimization of water supply systems

Reductions of water deficits for users and energy savings are frequently conflicting issues when optimizing largescale multi-reservoir and multi-user water supply systems. Undoubtedly, a high level of uncertainty due to hydrologic input variability and water demand behaviour characterizes these problems. The aim of this paper is to provide a decision support for the water system authority, in o...

Accelerated Gradient Methods for Stochastic Optimization and Online Learning

Regularized risk minimization often involves non-smooth optimization, either because of the loss function (e.g., hinge loss) or the regularizer (e.g., l1-regularizer). Gradient methods, though highly scalable and easy to implement, are known to converge slowly. In this paper, we develop a novel accelerated gradient method for stochastic optimization while still preserving their computational si...

Conditional gradient type methods for composite nonlinear and stochastic optimization

In this paper, we present a conditional gradient type (CGT) method for solving a class of composite optimization problems where the objective function consists of a (weakly) smooth term and a strongly convex term. While including this strongly convex term in the subproblems of the classical conditional gradient (CG) method improves its convergence rate for solving strongly convex problems, it d...